Bias-Variance Techniques for Monte Carlo Optimization: Cross-validation for the CE Method

In this paper, we examine the CE method in the broad context of Monte Carlo Optimization (MCO) [Ermoliev and Norkin, 1998, Robert and Casella, 2004] and Parametric Learning (PL), a type of machine learning. A well-known overarching principle used to improve the performance of many PL algorithms is the bias-variance tradeoff [Wolpert, 1997]. This tradeoff has been used to improve PL algorithms ranging from Monte Carlo estimation of integrals [Lepage, 1978], to linear estimation, to general statistical estimation [Breiman, 1996a,b]. Moreover, as described by Wolpert and Rajnarayan [2007], MCO is very closely related to PL. Owing to this similarity, the bias-variance tradeoff affects MCO performance, just as it does PL performance. In this article, we exploit the bias-variance tradeoff to enhance the performance of MCO algorithms. We use the technique of cross-validation, a technique based on the bias-variance tradeoff, to significantly improve the performance of the Cross Entropy (CE) method, which is an MCO algorithm. In previous work we have confirmed that other PL techniques improve the perfomance of other MCO algorithms [see Wolpert and Rajnarayan, 2007]. We conclude that the many techniques pioneered in PL could be investigated as ways to improve MCO algorithms in general, and the CE method in particular. The rest of the paper is organized as follows. In Sec. 2, we present an overview of the bias-variance tradeoff. In Sec. 3, we describe a few ways to exploit this tradeoff, starting from the relatively simple case of Monte Carlo integration, and proceeding to the more complex case of MCO. We also describe the original exploitation of this tradeoff, as a way to improve PL algorithms. In Sec. 4, we describe how to use cross-validation, a particular technique based on the bias-variance tradeoff, to modify the CE method. Sec. 5 then presents performance comparisons between this modified version of the CE method and the conventional CE method. These comparisons are on continuous, multimodal, unconstrained optimziation problems. We show that on these problems, using the modified version of the CE method can significantly improve optimization performance of the CE method, and never worsens performance.